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Abstract 



Automation of survey process has proved successful in many industries, yet it is still under- 
utilized in educational research. This is largely due to the fact that: (1) number crunching is 
usually carried out using software developed before information technology existed, and (2) the 
educational research is to a great extent trapped in its methods of producing tabulations, charts 
and reports. In this paper, we will demonstrate how we use automated survey data processing 
and reporting system, called AutoRide, to cut costs, increase turnaround and improve the quality 
of products delivered in both paper and online formats. With it, customized reports are 
automatically generated with faster and reliable results. 
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Introduction 



Automation has affected every part of survey process. Survey design, sampling, respondent 
selection, fieldwork administration, data capture, quality control, coding, editing, analysis, 
reporting and dissemination have all been influenced by technological changes. Computer 
assisted telephone interview (CATI), computer assisted personal interview (CAPI) and Web- 
based online survey make automated collection of large-scale surveys possible. Optical mark 
recognition (OMR) technology allows data capturing to be done quickly and efficiently. Recent 
development of internet technology makes the resultant data reported interactively on the Web. 

Automation of survey process has proved successful in many industries [Hearn, 2000], yet it is 
still under-utilized in educational research. This is largely due to the fact that: (1) number 
crunching is usually carried out using software developed before information technology existed, 
and (2) the educational research is to a great extent trapped in its methods of producing 
tabulations, charts and reports. In this paper, we will demonstrate how we use automated survey 
data processing and reporting system, called AutoRide, to cut costs, increase turnaround and 
improve the quality of products delivered in both paper and online formats. With it, customized 
reports are automatically generated with faster and reliable results. 

Background 

A large-scale survey (hereafter, the Survey) was developed to examine the degree to which a 
broad range of recommendations for effective school reform are implemented in a school as well 
as to examine more fully their impact on students and staff. It has provided much needed 
empirical data for most of the major foundation-funded initiatives in middle grades including the 
Carnegie Corporation’s Middle Grades School State Policy Initiative, the Lilly Endowment’s 
Middle Grades Improvement Project, and the Kellogg Foundation’s Middle Start Initiative. The 
survey has been extended to serve students in grades K-12 in other projects including the State of 
Rhode Island’s ongoing statewide improvement efforts. 



During the past 10 years, over 90,000 teachers and approximately one million students in over 
3000 schools have participated in the Survey. In the case of Rhode Island, the Survey was used 
as an integral part of statewide accountability system [Felner et al., 2001; Felner et al., 2000], 

The data collected from survey is summarized and presented in the format that helps schools 
understand their current status and the future goals. Over 300 public schools in Rhode Island 
participate in data collection each year. The data collected from teachers, students, administrators, 
and parents are tabled and charted via AutoRide and reported back to each school for their school 
improvement planning. As the reporting procedure includes a huge amount of information from a 
large number of schools, it is imperative that it is reliable and efficient. Time-sensitive survey 
data processing occurs during academic year. In most cases, we work with tight deadlines. In 
order to cover a wide range of schools with different building configuration, the procedure also 
needs to be flexible to be easily customized. 

Technical Issues 

AutoRide was developed using Visual Basic 6.0 with the OLE (Object Linking and Embedding) 
control [Balena, 1999]. OLE was used to run the application software by Visual Basic. The 
application software includes SPSS, Quattro Pro, WordPerfect and Harvard Graphics. Data 
Access Objects (DAO) in Visual Basic enables the user to access and manipulate data in local 
and remote databases created in Microsoft ACCESS format (see Figure 1). Macro functions in 
the application software are used to simplify Visual Basic programming and make the 
maintenance easy. Since Harvard Graphics does not support the OLE control in Visual Basic, 
SendKeys method in Visual Basic was used. SendKeys sends one or more keystrokes to the 
active window as if typed at the keyboard. SendKeys method is not safe and needs to have some 
knowledge of timing control. Extra time is helpful to ensure the task will be done in the time 
limit. 

Figures 2 illustrates the interaction between various components necessary for the AutoRide to 
work. School data information database is integrated into the system to provide the parameters 
such as school name, network number (the year joined), grade range, and project name. And the 
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file information database was created to include the file information used to process all templates 
and files. AutoRide is designed to be user-friendly with graphic interface (see Figures 3 and 4). 

Three running modes, interactive, batch, and easy batch modes are provided. In the interactive 
mode, a user interactively processes school by school. The user enters a school code, then select 
one of the application software to run (i.e., to run SPSS, to create WordPerfect tables, or to create 
Harvard Graphics charts). The user also selects which survey form to be used: teacher, student 
(elementary, middle, high, combined), TSRS (teacher-student rating scale), or parent surveys. In 
the batch mode, a user can process as many schools as possible. The user can select one or more 
options of running SPSS, WordPerfect or Harvard Graphics. Multiple types of survey forms can 
be selected (see Figure 5). In the easy batch mode, a user does not need to type school codes, to 
specify the application software, nor to specify survey forms. Once the user puts all raw data in 
one folder, AutoRide retrieves all the school codes in the folder and produce reports. This mode 
is very efficient when a large number of schools should be processed at once. AutoRide can 
process up to 100 schools per personal computer at one night, from running SPSS to creating all 
reports without any human presence. In addition to data reporting process, there are two 
additional programs developed to supplement AutoRide: verification of scanned data before 
reporting and PDF file extraction on the web. The scanned data verification checks the validity 
of input data by comparing this year’s data with last year’s data (min, max, mean values). It 
helps prevent errors in scanning and new survey design. After a hard copy report (over 150 
pages) is produced for each school, each page is converted into the PDF (portable document 
format) file for Web publishing using PDF file extraction program. This process is also 
automated. 

Benefits 

Over three year period, AutoRide has reduced a significant amount of manual process for large- 
scale educational surveys and improved the efficiency of the survey processing system and the 
quality of the products that are delivered in both paper and online formats. The scope of 
automated process that is considered includes data processing, customized documentation, and 
electronic dissemination. Not only were hours saved in staff time, but the automation was also 
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much more error-free. A huge manpower was put to run the data processing in previous years. 
By using the automated system, one person can run all of data processing in much less time 
period. The accuracy was also greatly improved. Typically, one staff was able to run 
approximately 10 schools per day manually. After automation was implemented, data from more 
than 500 schools were processed overnight. This demonstration will show how the automated 
data processing can help process the survey data more reliably and efficiently. Hands-on 
demonstration format would be beneficial for interactive discussions and presentations. 
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Figure 1. System flowchart of AutoRide 
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Figure 2. Procedure of AutoRide 
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Figure 3. Screen shot of AutoRide (#1) 
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Figure 4. Screenshot of AutoRide (#2) 
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Figure 5. Screenshot of AutoRide (#3) 
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